Image classification-dependent user interface in ultrasound imaging

ABSTRACT

For a classification-dependent user interface in ultrasound imaging with an ultrasound scanner, the ultrasound scanner classifies a view represented in an image. The user interface changes according to the view, allowing one or a few user inputs to be used for different user options or behavior combinations appropriate for the classified anatomy. The context from imaging alters the behavior of a given user input element of the user interface.

BACKGROUND

The present embodiments relate to ultrasound imaging. In ultrasoundimaging, the sonographer controls the scanning using various userinputs. This may require the sonographer to perform repetitive motionsand/or to reposition their hand to accomplish workflow tasks in controlof the imaging the patient. There is a need for an improved userinterface which minimizes the manual effort required to accomplishworkflow tasks.

Some ultrasound scanners have a user interface area that allows manyfunctions to be performed without moving the hand. For example, atrackball is surrounded by three or four labeled function keys, allowingthe user to control a cursor or pointer as well as perform basicfunctions such as update, position, and size. This user interface areamay minimize motion but has a limited number of function keys.Performing other workflow tasks often requires use of other keys spacedfrom the trackball user interface area.

SUMMARY

By way of introduction, the preferred embodiments described belowinclude methods, computer readable storage media, instructions, andsystems for a classification-dependent user interface in ultrasoundimaging with an ultrasound scanner. The ultrasound scanner classifies aview represented in an image. The user interface changes according tothe view, allowing one or a few user inputs to be used for differentuser options or behavior combinations appropriate for the classifiedanatomy. The context from imaging alters the behavior of a given userinput element of the user interface.

In a first aspect, a method is provided for a classification-dependentuser interface in ultrasound imaging with an ultrasound scanner. Theultrasound scanner scans a patient and classifies content of an imagefrom the scanning. Anatomy selection options are assigned to a userinput element of the user interface. The anatomy selection options areassigned based on the classified content. A Doppler gate or region ofinterest for flow imaging is placed in response to user input with theuser input element. The user input selects anatomy using the anatomyselection options as assigned based on the classified content. Flow isimaged with the Doppler gate or region of interest as placed.

In a second aspect, a method is provided for a classification-dependentuser interface in ultrasound imaging with an ultrasound scanner. Theultrasound scanner scans a cardiac region of a patient. The ultrasoundscanner identifies a cardiac view in an image from the scanning. A listof anatomical locations of the identified cardiac view is assigned to asingle user input of the ultrasound scanner. One of the anatomicallocations is selected in response to operation of the single user input.Flow imaging at the selected one of the anatomical locations isperformed.

In a third aspect, a system is provided for an imageclassification-dependent user interface in ultrasound imaging. An imageprocessor is configured to classify anatomy represented in ultrasounddata, alter behavior of a user interface component based theclassification of the anatomy such that different classificationprovides for different behavior of the same user interface component,and receive selection of an anatomy area using the user interfacecomponent with the behavior for the classification. An ultrasound systemis configured to image based on the selection of the anatomy area.

The present invention is defined by the following claims, and nothing inthis section should be taken as limitations on those claims. Furtheraspects and advantages of the invention are disclosed below inconjunction with the preferred embodiments and may be later claimedindependently or in combination.

BRIEF DESCRIPTION OF THE DRAWINGS

The components and the figures are not necessarily to scale, emphasisinstead being placed upon illustrating the principles of the invention.Moreover, in the figures, like reference numerals designatecorresponding parts throughout the different views.

FIG. 1 is a flow chart diagram of one embodiment of a method for aclassification-dependent user interface in ultrasound imaging with anultrasound scanner;

FIGS. 2A and 2B illustrate example ROI locations available based onclassification of a view as apical four chamber;

FIG. 3 illustrates an example of cycling through different anatomicallocations for a parasternal long axis view;

FIG. 4 illustrates an example of cycling through different anatomicallocations for an apical four chamber view;

FIGS. 5A and 5B illustrate example Doppler gate locations availablebased on classification of a view as apical four chamber;

FIG. 6 illustrates example graphics for selection of anatomy locations;and

FIG. 7 is a block diagram of one embodiment of a system for aclassification-dependent user interface in ultrasound imaging.

DETAILED DESCRIPTION OF THE DRAWINGS AND PRESENTLY PREFERRED EMBODIMENTS

An ultrasound scanner includes an image classification-dependent userinterface. Based on the live classification of one or more displayed ornon-displayed images, the function of user interface elements isdirected to specific behavior to achieve desired workflow actions withminimal hand motion. For example, a button in a homebase area (e.g.,trackball and immediately surrounding buttons) is assigned differentanatomical references for flow imaging based on the anatomical contentdetermined from classification. One or more user interface elementsbehave in a way that changes with image anatomy classification content.

In a flow imaging example, the user interface element behaves to selectdifferent color ROI placement options based on classification of theanatomy represented in an image. Different image views result in thesame user interface elements being associated with different selectableanatomy. The anatomy area for color region-of-interest (ROI) placementis selected using the programmed behavior. In another flow imagingexample, the user interface element enables selecting the anatomy areafor Doppler gate placement. The Doppler gate is placed in response tothe user-accepted, prior color Doppler ROI placement. Anatomy within oraround the ROI may be detected and used in placing the Doppler gate. Theuser interface element is assigned behavior based on classification ofthe image content, which behavior is used to place the ROI. The ROI isused to position the Doppler gate. Alternatively, the behavior of theuser interface element is set to cycle through different Doppler gatelocations based on the classification of the image anatomical content.

In one embodiment, a single button, softkey, command word, or other userinterface element is assigned content-based behavior. A list of anatomyassociated with the content is linked to the single user interfaceelement. Activation of that element cycles or advances through thesequence of anatomy (e.g., possible ROI or gate locations). One touch orvoice-initiated advancement within the sequence of anatomy areas isprovided as the behavior. Different content being recognized alters thebehavior of the single user interface element by using a different listof anatomical area options. A single keystroke may select a desiredcolor ROI and/or PW gate placement from a list of anatomy areapossibilities, which possibilities depend on the view being scanned.

The examples below use ultrasound imaging of a cardiac region of apatient. For example, four standard views are provided for cardiacimaging—apical four chamber, apical two chamber, parasternal long axis,and parasternal short axis. Image classification is used to recognizethe view. Based on the recognition of a particular view, the behavior ofa user interface element is set. Different views correspond to differentbehavior (e.g., different possible anatomical-related ROI or gatelocations and/or corresponding priority in the list). In otherembodiments, other behaviors than an ordered list, other types orregions of imaging than cardiac, and/or other views than the fourstandard views are provided. For example, the ultrasound systemclassifies a view relative to a fetus or particular anatomy in the view.The user interface element is then programmed to provide workflowoperations associated with that particular anatomy or view.

FIG. 1 shows one embodiment of a method for a classification-dependentuser interface in ultrasound imaging with an ultrasound scanner. Theultrasound scanner identifies the view or other anatomical contentrepresented by ultrasound data as a classification. The classificationis used to alter operation of one or more user interface elements, suchas assigning a list of anatomy-related placements of ROIs and/or Dopplergates to a single user interface element where different lists are usedfor different classes.

The method is performed by the system shown in FIG. 7 or a differentsystem. For example, a medical diagnostic ultrasound imaging systemscans in act 10 and images in act 15. An image processor classifies inact 11, detects in act 12, assigns in act 13, and places in act 14. Auser interface, such as a button or key, is used with or by the imageprocessor for the assigning of act 13 and the placement of act 14. Adisplay is used for imaging in act 15. The display may be used to showthe user placement in act 14.

The acts are performed in the order shown (i.e., top to bottom ornumerical) or another order. For example, act 12 is performed as part ofact 11 or performed before act 11 to assist in classification. Asanother example, act 12 is performed after the assignment of act 13.

Additional, different or fewer acts may be used. For example, acts forconfiguring the ultrasound scanner to perform imaging are included. Asanother example, acts for the review or another use of the image areperformed. In yet another example, act 15 is not performed.

In act 10, the ultrasound scanner scans a patient. For example, theultrasound scanner scans a cardiac region (e.g., heart). Acoustic energyis transmitted, and echoes are received. Any type and/or format ofscanning is provided. For example, a B-mode scan is performed. B-modeframes of data are generated by B-mode scanning. A B-mode imagerepresents the intensity or strength of return of acoustic echoes. Othertypes of detection and corresponding scans are performed. For example,color flow (e.g., flow-mode) estimation is used. Velocity, power, and/orvariance as a function of location are estimated. As another example,harmonic mode is used, such as imaging at a second harmonic of afundamental transmit frequency. Combinations of modes may be used. Inyet another example, a Doppler or PW mode is used, such as for spectralimaging.

The scanning is to acquire a frame of data representing the patient at agiven period or time. In other embodiments, the scanning is on-going. Asequence of frames of data are acquired over time. The frames of dataare in a scan format, such as a polar coordinate format. Alternatively,the frames of data are scan converted into a display format, such as aCartesian coordinate format. The ultrasound data is data after or beforebeamformation, detection, filtering, scan conversion, display colormapping, or display. An ultrasound image may be a frame of data that maybe used for imaging (e.g., beamformed or detected data prior to scanconversion) or has been used for imaging (i.e., displayed or formattedfor display).

In act 11, the ultrasound scanner classifies the content of an image. Aframe of ultrasound data or another image representing the patient isclassified. The classification occurs in real-time, such as before theultrasound scanner completes acquisition and/or generation of anotherimage and/or within 1 or 2 seconds of having acquired the frame ofultrasound data. Alternatively, the classification is performed off-lineor not in real-time.

The classification identifies a view or anatomical content. In thecardiac imaging example, the classification identifies the image asrepresented an apical four chamber, an apical two chamber, a parasternallong axis, or a parasternal short axis view. In other examples, theclassification identifies the image as representing a given organ orregion of the body, such as classifies the ultrasound data asrepresenting the heart.

Any classification may be used. For example, pattern or templatematching is used. The template that best matches the distribution ofultrasound data is labeled. The label is the classification. As anotherexample, a machine-learned model, such as a neural network, classifies.The ultrasound data is input to the machine-learned model, which outputsthe classification in response. Other image processing may be used forclassification.

The classification provides information about what is represented in theimage. This information may be used to adapt the user interface. Ratherthan requiring user input or indication by what imaging application isselected and further combinations of inputs to establish the desiredworkflow, the ultrasound scanner identifies the anatomical content anduses the information to program the user interface.

In act 12, the ultrasound scanner detects locations of features of thecontent. As part of classification or post-classification, one or morelandmarks, organs, or other locations are detected. For example, valve,vein, vessel, artery, heart wall, and/or other cardiac structures aredetected. The location or locations within the region (e.g., field ofview (FOV)) represented by the ultrasound data for one or more featuresare determined.

The features to be detected and/or the detectors to use for detectionmay be selected based on the classification. Alternatively, detectorsare applied regardless of the classification. The features may be usedto find different anatomical regions (areas) that make up the anatomicalcontent represented in the frame of ultrasound data.

Any detector may be used. For example, region-growing, skeletonization,and/or other image processing is used. As another example, templatematching is performed. In yet another example, a machine-learned modelor models are applied.

In act 13, the ultrasound scanner (e.g., image processor) assignsbehavior to the user interface element based on the classification. Forexample, the anatomy selection options are assigned to a user inputelement of the user interface.

The user interface includes one or more user input devices. For example,the user interface includes one or more buttons or keys, sliders,rotating knobs, touch pads, track ball, mouse, and/or touch screen. Thebuttons or keys may be part of a keyboard or separate from a keyboard.In one embodiment, a user control console includes a trackball with oneor more buttons surrounding or adjacent to the trackball. These buttonsmay be labeled for particular purposes, such as update, enter/select,position, and/or size. The buttons are against or adjacent to thetrackball. Other buttons are spaced further from the trackball. A voicecommand word may be used as the user interface element.

The function or behavior of one or more user interface elements (e.g.,entry or control device) is assigned. The behavior may or may not matchthe labeled function. One or more workflow options are linked to theuser interface element. For example, a list of anatomy is linked. Theuser interface element is assigned to cycle through the list in anorder. Anatomy selection options are assigned to the user interfaceelement. In other embodiments, other options than anatomy selectionoptions are linked. For example, options to activate, alter type ofdetection, change filtering, adjust amplification, and/or other steps ina workflow implemented by the ultrasound scanner are linked. Any ofvarious behaviors may be linked.

In one embodiment, the assignment is for a single one of the userinterface elements. For example, one button (e.g., “update” button) isassigned the anatomy selection options. By depressing that button,different anatomy options are provided for selection. The order ofpresentation for selection (i.e., the order of anatomy presented as wellas which anatomy) is used to link input using the user interface elementto operation of the ultrasound scanner.

The anatomy selection options or other options are assigned based on theclassified content. In the anatomy selection options embodiment, theclassified content identifies a list of anatomy viewable or representedin the image. Different anatomical content (i.e., differentclassification) results in different lists of anatomy for selection. Thedifferent anatomy likely to be of interest for a given view is includedin the list for selection by the user. The behavior (e.g., list ofoptions) changes for the user interface element based on the content.

The list includes the locations of the features in the image. Based onthe classification in act 11 and/or detection in act 12, a list ofanatomical selection options for anatomy or features (e.g., landmarks)represented in the ultrasound data is assigned to the user interfaceelement. In the cardiac example, an ordered list of anatomical locationsin the identified cardiac view are assigned to one or more userinterface elements, such as assigning the list to a single user input(e.g., “update” button). The list of anatomy areas is dependent on theactual acquired image contents determined by automatic viewclassification.

The classification in act 11 and the assignment of act 13 occur inreal-time with the scanning of act 10. As imaging continues, theclassification and assignment also continue. The assignment may beupdated where the content changes. Where the content continues to be thesame as confirmed by the last classification, the assignment stays thesame. Within 1 or 2 seconds of completion of acquisition or imagegeneration of a current frame of data and/or before formation of thenext image or frame of ultrasound data, the classification is repeated,and the assignment is confirmed or altered. For example, the assignmentoccurs based on the view classifier after (or before) entry into coloror pulse width (PW) modes.

FIGS. 2A and 2B show an example. The classification of the image ofFIGS. 2A and 2B is apical four chamber view. Based on thisclassification, two anatomical regions or areas 22, 24 are assigned to abutton (e.g., “update” button on a control console). These two areas 22,24 represent areas that may be of interest in this view. The areas 22,24 are regions of interest (ROIs) fitting around, over, or withindetected anatomy. For example, the areas 22, 24 are color ROIs. Theclassification causes the assignment of these two ROIs 22, 24 to theuser interface element, then allowing the user to select the anatomy ofinterest for this scan of this patient.

The user input or interface element may be a button, such as a key. Inone embodiment, one of the “generic” keys (e.g., “update” button)adjacent to a trackball or other cursor control is used. In otherembodiments, a button on a transducer (e.g., a multi-function “+” key onthe transducer) is used. By operating the button, the selection optionsassigned to the element are cycled.

In another embodiment, the user input or interface element is a softkeyor region of a touch screen. A graphical representation of the anatomyselection options assigned based on the classified content is generated.A touch screen area displays ROI placement anatomy area choices. A heartrepresentation, such as a heart icon or icons of different anatomyforming a cartoon, wire frame, or graphical representation of thecontent of the image may be used. In another embodiment, a live actualcardiac image with areas highlighted is used. The touch screen userinterface element shows and receives selection of the different detectedanatomy options appropriate for the classified view.

FIG. 6 shows an example. The classification indicates a parasternal longaxis view, so a list of anatomy options for that view are presented. Thelist includes the mitral valve, the apical valve, the left ventricle,and a combination of the apical valve and the mitral valve. Where theclassification is the apical four chamber view, the options displayedmay be tricuspid valve and mitral valve. Additional, different, or feweranatomical locations may be included in the lists of anatomicallocations of these examples. Other views may have other lists.

In another embodiment, the user interface element is a voice activationterm. For example, the word “next” or “update” is assigned to the listof anatomical selection options. The user utters the word “next” or“update,” and, depending on the context, the system provides varyingbehavior (e.g., different anatomical selection options or order). Thevoice activation is provided with image class dependent context. In oneembodiment, B-mode imaging is performed. The view is classified as oneof the standard cardiac views. The user utters “color,” which places anROI at a first anatomy area of the options. If in a color imaging mode,uttering “color” may exit the color mode. If in a PW mode, uttering“color” may enter the color mode. Similarly, uttering PW or Doppler mayplace a gate at the first anatomical location in the list if in B-mode,exit color and enter PW mode if in color mode, or exit PW mode if in PWmode. Uttering “next” or “update” in B-mode may provide no action or mayplace a color ROI without activating color mode. Uttering “next” or“update” in color or PW modes may move the ROI or gate, respectively, tothe next anatomy option on the list, allowing the user to operate theone user interface element to cycle through various anatomical optionsfor ROI or gate placement associated with the view as classified. Othercommand words and/or actions may be used.

The standard views may depend on the type of ultrasound imaging and/ortransducer, such as trans-thoracic echo, trans-esophageal, and intracardiac echo. Standard views for trans-thoracic echo include Apical2,Apical3, Apical4, Apical5, Parasternal Long Axis, Parasternal Long AxisRight Ventricular Inflow Tract, Parasternal Long Axis Right VentricularOutflow Tract, Parasternal Short Axis Aortic Valve, Parasternal ShortAxis Left Ventricle, Parasternal Short Axis Mitral Valve, Subcostal FourChamber, Subcostal Inferior Vena Cava, and Suprasternal. Standard viewsfor trans-esophageal include Mid-Esophageal (ME) views, such as ME 2Chamber, ME 4 Chamber, ME_AOV_SAX, ME_LAA, and ME_LAX. Standard viewsfor intra cardiac echo may include Left Atrium, Left Atrial Appendage,Left Inferior Pulmonary Vein, Right Inferior Pulmonary Vein, LeftSuperior Pulmonary Vein, Right Superior Pulmonary Vein, Esophagus, andInteratrial Septum. In non-cardiac examples, standard views may includeLeft Kidney longitudinal, Left Kidney short axis, Right Kidney, andLiver.

In act 14 of FIG. 1, the image processor or ultrasound scanner operatesbased on user input with the assigned user interface element. In theanatomical selection options embodiment, the image processor places anROI or PW gate based on operation of the user interface element.Subsequent operation cycles through the list of anatomical options.

In one embodiment, the assigned behavior includes turning on flowimaging (e.g., color or flow mode representing spatial distribution ofvelocity, energy, and/or variance or PW or Doppler mode representingspectral information at a gate). The use or operation of the userinterface element turns on flow imaging as well as places the ROI orgate within the B-mode image. The user input selects the anatomy ofinterest for flow imaging and activates the flow imaging. The anatomy ofinterest is selected from the options based on the classified content.By repeating user input or operation of the user interface element, theanatomy of interest for the flow imaging shifts or changes to the nextROI or gate location on the list. A flow region of interest for the flowimaging is cycled through the anatomy list in response to use of theuser input element (e.g., operation of the button). For example, the“update” key cycles through a set of anatomy areas for targetingautomatic ROI placement. The sequence depends on the view classifierresult. If the view classifier sees a new view, then the anatomy areasequence is reset or returns to a previously stored state for the givenanatomy

The ROI is a scan region within a field of view (FOV). For example, theROI is a sub-set of the FOV. The ROI is shaped based on the scan linedistribution. For linear scans, the scan lines are parallel. Theresulting ROI is a square or rectangular box. For sector or Vector®scans, the scan lines diverge from a point on the transducer face or avirtual point positioned behind the transducer, respectively. The sectorand Vector scan formats of scan lines scan in a fan shaped ROI. TheVector scan may be a fan shaped region without the origin pointincluded, such as resembling a trapezoid (e.g., truncated triangle).Other shapes of ROIs may be used, such as square or rectangular in asector or Vector® scan.

The orientation may also be determined to include or avoid certainlocations. The orientation may be based on the limits on steering from atransducer, detected landmarks that may cause acoustic shadowing, and/ordirectivity response of the tissue being quantified.

The ROI has a default size. The ROI is any size, such as 3 cm in lateraland 5 cm in axial. For flow imaging, the ROI is sized to avoid tissuelocations. The size may be based on the locations of detected anatomy,fitting the ROI to the patient.

For a PW gate, the gate is sized and positioned to be in the flow ofinterest. For example, the PW gate is placed over a valve, in the inflowtract of the valve, or in the outflow tract of a valve. The PW gate is apoint or region (e.g., includes one or more beamformer sample locations)where the measurements are combined for Fourier transformation togenerate a spectrum representing energy as a function of frequency orvelocity at a given period or time.

By operating the user input element, the assigned behavior is activated.In the embodiment of FIG. 1, the assigned behavior is the list ofanatomy selection options. For example, one of the anatomical locationsis selected in response to operation of the single user input. Byoperating the single user input again, the next anatomical location inthe list of options based on the classification is selected and thecorresponding flow region placed based on the selection. The user maycycle through the list in response to repetitive operation of the singlebutton. The list of anatomy areas is dependent on the actual acquiredimage contents determined by automatic view classification. Initialentry to color or PW mode will pick the first anatomical area in thesequence based on view. Subsequent activation or use of the userinterface element selects the next anatomical area. For example, eachpress of the update key initiates inference (i.e., classification) andROI placement, cycling to the next anatomy area target.

FIGS. 2A and 2B show one example. The classification identifies the viewas an apical four chamber view. The first anatomical region on the listassigned to the user interface element for this view is an ROI 24 aroundthe mitral valve. When the user operates the user interface element, theROI 24 is positioned by the image processor based on the detection, andcolor or flow mode is activated for imaging flow in the ROI 24 (see FIG.2A). If the user operates the user interface element again, the ROI 24is shifted to the ROI 22 for color or flow imaging of the tricuspidvalve (see FIG. 2B). Operation again would shift to the next anatomicalselection option in the assigned list. For example, operation againshifts back to the ROI 24.

FIG. 3 shows an example from the user interface perspective. A trackball with three adjacent buttons is represented. The upper button 30 isan “update” button but may have a different label. The view isclassified as the parasternal long axis view. The initial anatomicalregion for color or flow-mode imaging is an ROI for imaging flow at boththe mitral and atrial valves. Activating the button 30 causes the ROI tochange to the left ventricle. Activating the button 30 again causes theROI to change to the mitral valve. Activating the button 30 yet againcauses the ROI to change to the atrial valve. Activating the button 30another time causes the ROI to change back to the ROI for both themitral and atrial valves.

FIG. 4 shows an example corresponding to the apical four chamber view ofFIGS. 2A and 2B. The button 30 is activated to place the ROI for imagingflow at the mitral valve. Activating the button 30 again causes the ROIto change to the tricuspid valve. Activating the button 30 yet againcauses the ROI to change back to the mitral valve.

FIGS. 5A and 5B show an example for placement of a PW or Doppler gate 50for the flow imaging. In one embodiment, the anatomical selectionoptions assigned to the user interface element are gate locations. Byactivating the user interface element, the next gate location on thelist of locations for the recognized view is used. In this apical fourchamber view example, two gate locations are provided—the mitral valveand the pulmonic vein. The Doppler gate 50 is placed for the flowimaging by cycling through the anatomy list in response to use of theuser input element.

In another embodiment, the PW or Doppler gate 50 positioning is tied tothe ROI selection. The selection of the ROI 22, 24 also providesselection of a corresponding PW gate 50 location. The Doppler gate 50 isplaced based on selection of anatomy for a flow ROI 22, 24 using theanatomy selection options. Automatic PW gate placement depends on thecardiac view with anatomical area selection dependent on the currentcolor ROI position and its correspondence to detected anatomy. Forexample, use of the user interface element to select the ROI 24 over themitral valve also places the PW gate 50 at the mitral valve. Furtheractivation of the user interface element shifts the ROI to the ROI 22,which places the PW gate 50 at the pulmonic vein. In other embodiments,activation of the user input element may shift or change position of thePW gate 50 for a given ROI. For example, the list includes combinationsof ROI and PW gate positions. Each activation moves to the nextcombination. In another embodiment, the Doppler gate is place first.When the user activates color (flow-mode), the flow ROI is positionedbased on the selection of anatomy for the Doppler gate (i.e., based onthe Doppler gate location). The PW gate placement guides the selectionof a color ROI. If the user is operating in PW, then entry to colorplaces the color ROI according to the view classifier while selectingthe anatomy area most closely associated with the PW gate.

In act 15 of FIG. 1, the ultrasound scanner images based on the selectedoptions from the behavior. In response to operation of the userinterface element, the assigned action is performed by the ultrasoundscanner. The user does not have to navigate to controls specific forsequencing through the actions (e.g., does not have to manually placethe ROI and activate). Instead, the context-specific assigned optionsare used to automatically progress through the workflow.

In the cardiac example, flow imaging is performed. The flow imaging isfor the ROI or PW gate as selected using context-based user interface.The selection both places the ROI or PW gate as well as activates theimaging. Alternatively, the placement occurs, and then subsequentoperation of the user interface element or another element activates theimaging. The flow at a currently selected one of the anatomy selectionoptions is imaged. The flow at other anatomy options may be imaged bychanging the placement (i.e., activating the user interface element oneor more times). Spectral Doppler (i.e., Doppler or PW mode) and/or colorDoppler or flow-mode imaging is performed for the Doppler gate and/orflow ROI, respectively.

The flow imaging continues for any number of frames of ultrasound dataor period. The detection and corresponding location of the ROI or PWgate may be updated during the flow imaging. As the transducer and/orpatient shifts, the anatomy of interest may shift. By repeating thedetection of anatomy, the location of the ROI or PW gate for theselected anatomy may automatically shift with the anatomy.

The classification of act 11 and assignment of act 13 may be repeated.For example, every time the user operates the user input element, theclassification and assignment are repeated. If the view is the samebased on the classification, then the assignment stays the same. If theview changes, then the assignment may change to the behavior (e.g., listof anatomy selection options) for the current or new view. For example,pressing “update” when in color initiates the view classification andobject detection inference sequence. If the inferred cardiac view isunchanged, then object detection will select the next anatomical area inthe sequence. If the view changes, then the anatomical area sequencestarts back at the beginning for that view. The anatomy selectionoptions assigned to the user input element vary with variance in thecontent as classified.

Once the ROI or PW gate is positioned or operation of the scannerchanged, the ultrasound scanner performs the imaging, such as flowimaging. The imaging results in formation of one or more images. In thecardiac example, the images are combination B-mode with color modeand/or PW mode images. The generated image is displayed on a displaydevice.

The ultrasound imaging is used for diagnosis, prognosis and/or treatmentguidance. The use of classification-dependent user interface may avoidrepetitive motions, simplify operation, and/or require a lesser extentof motion in operating the ultrasound scanner. The sonographer andpatient benefit from the improvement as the total scan time may be lessand more consistent imaging may be provided.

FIG. 7 shows one embodiment of a system 70 for an imageclassification-dependent user interface in ultrasound imaging. Thesystem 70 is an ultrasound imaging system where the actions from one ormore user inputs are assigned based on classification of the content inthe imaging.

The system 70 is an ultrasound imager or scanner. In one embodiment, theultrasound scanner is a medical diagnostic ultrasound imaging system. Inalternative embodiments, the system 70 is a personal computer,workstation, PACS station, or other arrangement at a same location ordistributed over a network for real-time or post acquisition imaging.

The system 70 implements the method of FIG. 1 or other methods. Thesystem 70 includes a transmit beamformer 71, a transducer 72, a receivebeamformer 73, an imager former 74, a display 75, an image processor 76,and a user input 77. Additional, different or fewer components may beprovided. For example, a spatial filter, a scan converter, a mappingprocessor for setting dynamic range, and/or an amplifier for applicationof gain are provided. As another example, the user input 77 is part ofthe display 75, such as where a touch screen display is used.

The transmit beamformer 71 is an ultrasound transmitter, memory, pulser,analog circuit, digital circuit, or combinations thereof. The transmitbeamformer 71 is configured to generate waveforms for a plurality ofchannels with different or relative amplitudes, delays, and/or phasingto focus a resulting beam at one or more depths. The waveforms aregenerated and applied to a transducer array with any timing or pulserepetition frequency.

The transmit beamformer 71 connects with the transducer 72, such asthrough a transmit/receive switch. Upon transmission of acoustic wavesfrom the transducer 72 in response to the generated waves, one or morebeams are formed during a given transmit event. The beams are forB-mode, color or flow-mode, PW mode, or other mode of imaging. Sector,Vector®, linear, or other scan formats may be used. The same region isscanned multiple times for generating a sequence of images or forquantification.

The transducer 72 is a 1-, 1.25-, 1.5-, 1.75- or 2-dimensional array ofpiezoelectric or capacitive membrane elements. The transducer 72includes a plurality of elements for transducing between acoustic andelectrical energies. For example, the transducer 72 is a one-dimensionalPZT array with about 64-256 elements. As another example, the transducer72 is a transesophageal echocardiography (TEE) array, a volumeintracardiac echocardiography (ICE) array, or a trans-thoracic echo(TTE) array.

The transducer 72 is releasably connectable with the transmit beamformer71 for converting electrical waveforms into acoustic waveforms, and withthe receive beamformer 73 for converting acoustic echoes into electricalsignals. The transducer 72 transmits the transmit beams where thewaveforms have a frequency and are focused at a tissue region orlocation of interest in the patient. The acoustic waveforms aregenerated in response to applying the electrical waveforms to thetransducer elements. The transducer 72 transmits acoustic energy andreceives echoes. The receive signals are generated in response toultrasound energy (echoes) impinging on the elements of the transducer72.

The transducer 72 is a hand-held probe for use external to the patient.Alternatively, the transducer 72 is part of a probe for insertion withinthe patient. The transducer 72 may be positioned at various locationsrelative to the patient by the user and/or by a robotic arm.

The receive beamformer 73 includes a plurality of channels withamplifiers, delays, and/or phase rotators, and one or more summers. Eachchannel connects with one or more transducer elements. The receivebeamformer 73 applies relative delays, phases, and/or apodization toform one or more receive beams in response to each transmission fordetection. Dynamic focusing on receive may be provided. The receivebeamformer 73 outputs data representing spatial locations using thereceived acoustic signals. Relative delays and/or phasing and summationof signals from different elements provide beamformation.

The receive beamformer 73 may include a filter, such as a filter forisolating information at a second harmonic or other frequency bandrelative to the transmit frequency band. Such information may morelikely include desired tissue, contrast agent, and/or flow information.In another embodiment, the receive beamformer 73 includes a memory orbuffer and a filter or adder. Two or more receive beams are combined toisolate information at a desired frequency band, such as a secondharmonic, cubic fundamental, or another band. The fundamental frequencyband may alternatively be used.

The receive beamformer 73 outputs beam summed data representing spatiallocations. The beam summed data is in an I/O or RF format. Ultrasoundsignals or data are output.

The imager former 74 detects, such as detecting intensity, from thebeamformed samples. Any detection may be used, such as B-mode, Doppler(erg, PW Doppler), and/or color or flow detection. The imager former 74may or may not include a scan converter and/or spatial or temporalfilters.

The user input 77 is a mouse, trackball, touchpad, touch screen,keyboard, buttons, sliders, knobs, and/or other input device. The userinput 77 operates with the display 75 to provide a user interfacegenerated by the imager processor 76.

The image processor 76 is a controller, general processor, applicationspecific integrated circuit, field programmable gate array, graphicsprocessing unit, or another processor to control the user interface andoperation of the ultrasound system 70 based on classification of contentof an image. The imager processor 76 includes or interacts withdifferent components of the system 70 to control scanning and imaging.The imager processor 76 is configured by hardware, software, and/orfirmware.

The imager processor 76 is configured to classify anatomy represented inultrasound data. The content of one or more frames of ultrasound dataare identified automatically or without user indication for theparticular image.

The image processor 76 is configured to alter behavior of a userinterface component based the classification of the anatomy such thatdifferent classification provides for different behavior of the sameuser interface component. The action or actions assigned to one or morebuttons, keys, knobs, sliders, or other user interface component aredifferent depending on the identified content. For example, differentlocations and/or orders of locations for ROI or PW gate positioning areassigned to a single user interface component. Different content resultsin a different list and/or order being assigned.

The image processor 76 is configured to receive selection of an anatomyarea using the user interface component with the assigned behavior forthe classification. The operation of the user interface component, suchas a single button, activates the assigned action or actions. Forexample, operation of the single button cycles or sequences throughdifferent anatomical regions for the ROI or PW gate placement. Theassigned list of anatomical regions of the anatomy of the patientrepresented in the image or frame of ultrasound data is used to sequencethrough different placements. The classification indicates the availableplacements, and the assigned user interface element allows selection ofone of the available placements. For example, the location for a colorDoppler ROI or a Doppler gate for PW Doppler imaging is selected by theuser using the button.

The ultrasound system 70 is configured to image based on the selectionof the anatomy area. The color mode, flow-mode, or PW mode flow imagingis performed for the selected ROI and/or PW gate location. The ROIand/or PW gate selected from available locations for the anatomicalcontent of the image at a given time is used for imaging.

The display 75 is a CRT, LCD, monitor, plasma, projector, printer orother device for displaying an image or sequence of images. Any nowknown or later developed display 75 may be used. The display 75 displaysa B-mode image, a flow-mode image, a PW image, or another image. Thedisplay 75 may display one or more images representing the ROI or PWgate placement as part of the user interface for selecting the anatomyof current interest.

The imager processor 76 and/or the ultrasound system 70 operate pursuantto instructions stored in a memory. The instructions configure thesystem for performance of the acts of FIG. 1. The instructions configurefor operation by being loaded into a controller, by causing loading of atable of values (e.g., elasticity imaging sequence), and/or by beingexecuted. The memory is a non-transitory computer readable storagemedia. The instructions for implementing the processes, methods and/ortechniques discussed herein are provided on the computer-readablestorage media or memories, such as a cache, buffer, RAM, removablemedia, hard drive or other computer readable storage media. Computerreadable storage media include various types of volatile and nonvolatilestorage media. The functions, acts, or tasks illustrated in the figuresor described herein are executed in response to one or more sets ofinstructions stored in or on computer readable storage media. Thefunctions, acts or tasks are independent of the particular type ofinstructions set, storage media, processor or processing strategy andmay be performed by software, hardware, integrated circuits, firmware,micro code and the like, operating alone or in combination. Likewise,processing strategies may include multiprocessing, multitasking,parallel processing, and the like. In one embodiment, the instructionsare stored on a removable media device for reading by local or remotesystems. In other embodiments, the instructions are stored in a remotelocation for transfer through a computer network or over telephonelines. In yet other embodiments, the instructions are stored within agiven computer, CPU, GPU or system.

While the invention has been described above by reference to variousembodiments, it should be understood that many changes and modificationscan be made without departing from the scope of the invention. It istherefore intended that the foregoing detailed description be regardedas illustrative rather than limiting, and that it be understood that itis the following claims, including all equivalents, that are intended todefine the spirit and scope of this invention.

I (we) claim:
 1. A method for a classification-dependent user interfacein ultrasound imaging with an ultrasound scanner, the method comprising:scanning a patient with the ultrasound scanner; classifying, by theultrasound scanner, content of an image from the scanning; assigninganatomy selection options to a user input element of the user interface,the anatomy selection options assigned based on the classified content;placing a Doppler gate or region of interest for flow imaging inresponse to user input with the user input element, the user inputselecting anatomy using the anatomy selection options as assigned basedon the classified content; and imaging flow with the Doppler gate orregion of interest as placed.
 2. The method of claim 1 whereinclassifying and assigning occur in real-time with the scanning.
 3. Themethod of claim 1 wherein classifying comprises identifying a viewrepresented by the image.
 4. The method of claim 3 wherein identifyingthe view comprises identifying the view as one of apical four chamber,apical two chamber, parasternal long axis, and parasternal short axis.5. The method of claim 1 wherein classifying comprises classifying byapplication of a machine-learned model.
 6. The method of claim 1 whereinassigning comprises assigning an anatomy list as the anatomy selectionoptions for anatomy of the content of the image, and wherein placingcomprises placing the region of interest for the flow imaging by cyclingthrough the anatomy list in response to use of the user input element.7. The method of claim 1 wherein assigning comprises assigning ananatomy list as the anatomy selection options for anatomy of the contentof the image, and wherein placing comprises placing the Doppler gate forthe flow imaging by cycling through the anatomy list in response to useof the user input element.
 8. The method of claim 1 wherein assigningcomprises assigning an anatomy list as the anatomy selection options foranatomy of the content of the image, and wherein placing comprisesplacing the Doppler gate or the region of interest based on selectionfor the region of interest or the Doppler gate, respectively, using theanatomy selection options.
 9. The method of claim 1 further comprisingdetecting locations of features of the content, and wherein assigningcomprises including the locations of the features in the anatomyselection options.
 10. The method of claim 1 wherein the user inputelement comprises a button, and wherein placing comprises cyclingthrough the anatomy selection options in response to operation of thebutton, and wherein imaging comprises imaging the flow at a currentlyselected one of the anatomy selection options.
 11. The method of claim 1further comprising repeating the classifying and assigning in responseto use of the user input element.
 12. The method of claim 11 wherein theanatomy selection options assigned to the user input element vary withvariance in the content as classified.
 13. The method of claim 1 whereinassigning anatomy selection options to a user input element of the userinterface comprises a graphical representation of the anatomy selectionoptions assigned based on the classified content.
 14. The method ofclaim 1 wherein the user input element comprises a voice activationterm, and wherein assigning comprises assigning the anatomy selectionoptions to the voice activation term.
 15. A method for aclassification-dependent user interface in ultrasound imaging with anultrasound scanner, the method comprising: scanning, by the ultrasoundscanner, a cardiac region of a patient; identifying, by the ultrasoundscanner, a cardiac view in an image from the scanning; assigning a listof anatomical locations of the identified cardiac view to a single userinput of the ultrasound scanner; selecting one of the anatomicallocations in response to operation of the single user input; and flowimaging at the selected one of the anatomical locations.
 16. The methodof claim 15 wherein assigning comprises assigning the list to a singlebutton, and wherein selecting comprises cycling through the list inresponse to repetitive operation of the single button.
 17. The method ofclaim 15 wherein selecting comprises selecting for a Doppler gate or aflow region of interest, and wherein flow imaging comprises spectralDoppler or color Doppler imaging for the Doppler gate or the flow regionof interest, respectively.
 18. A system for an imageclassification-dependent user interface in ultrasound imaging, thesystem comprising: an image processor configured to classify anatomyrepresented in ultrasound data, alter behavior of a user interfacecomponent based the classification of the anatomy such that differentclassification provides for different behavior of the same userinterface component, and receive selection of an anatomy area using theuser interface component with the behavior for the classification; andan ultrasound system configured to image based on the selection of theanatomy area.
 19. The system of claim 18 wherein the user interfacecomponent comprises a single button where the behavior comprises a listof anatomical regions of the anatomy, the anatomy area comprising one ofthe anatomical regions where the list is based on the classification andoperation of the single button cycles through the list.
 20. The systemof claim 18 wherein the image processor is configured to receive theselection as a region of interest for color Doppler or a Doppler gatefor pulsed wave Doppler imaging.